Papers with dependency parsing
PhoNLP: A joint multi-task learning model for Vietnamese part-of-speech tagging, named entity recognition and dependency parsing (2021.naacl-demos)
Copied to clipboard
| Challenge: | PhoNLP is a multi-task learning model for joint Vietnamese part-of-speech (POS) tagging, named entity recognition (NER) and dependency parsing. |
| Approach: | They propose a multi-task learning model for Vietnamese part-of-speech tagging, named entity recognition and dependency parsing that fine-tunes the pre-trained Vietnamese language model PhoBERT for each task independently. |
| Outcome: | The proposed model outperforms a single-task learning approach that fine-tunes the pre-trained Vietnamese language model PhoBERT for each task independently. |
Evaluating zero-shot transfers and multilingual models for dependency parsing and POS tagging within the low-resource language family Tupían (2022.acl-srw)
Copied to clipboard
| Challenge: | Existing studies on NLP applications for low-resource languages have not been done in this area. |
| Approach: | They propose to replicate the transferability of dependency parsers and POS taggers trained on closely related languages within the low-resource language family Tupan. |
| Outcome: | The proposed models replicate the transferability of dependency parsers and POS taggers trained on closely related languages within the low-resource language family Tupan. |
Spa: On the Sparsity of Virtual Adversarial Training for Dependency Parsing (2022.findings-aacl)
Copied to clipboard
| Challenge: | Virtual adversarial training (VAT) is a powerful approach to improving robustness and performance, leveraging both labeled and unlabeled data to compensate for the scarcity of labeles. |
| Approach: | They propose a Sparse Parse Adjustment algorithm which combines VAT and a graph-based dependency parsing model in an exact computational manner and enhances the dependency parsed with controllable and adjustable sparsity. |
| Outcome: | Empirical results show that the proposed algorithm outperforms other methods without sparsity regularization. |
Building Hierarchically Disentangled Language Models for Text Generation with Named Entities (2020.coling-main)
Copied to clipboard
| Challenge: | Named entities pose a unique challenge to traditional methods of language modeling. |
| Approach: | They propose a Hierarchically Disentangled Model for named entities in cooking recipes using a dataset from several publicly available online sources. |
| Outcome: | The proposed model is based on 158,473 cooking recipes from public sources. |
Deep RNNs Encode Soft Hierarchical Syntax (P18-2)
Copied to clipboard
| Challenge: | Existing studies show that syntactic information is useful for a wide variety of NLP tasks. |
| Approach: | They propose to use word-level representations to learn internal representations that capture soft hierarchical notions of syntax from highly varied supervision. |
| Outcome: | The proposed model encodes significant amounts of syntax even without explicit supervision. |
TurkishDelightNLP: A Neural Turkish NLP Toolkit (2022.naacl-demo)
Copied to clipboard
| Challenge: | a neural Turkish NLP toolkit performs computational linguistic analyses from morphological level to semantic level. |
| Approach: | They propose a neural Turkish NLP toolkit that performs computational linguistic analyses from morphological level to semantic level. |
| Outcome: | The proposed toolkit performs computational linguistic analyses from morphological level to semantic level in Turkish. |
Syntax-aware Multi-task Graph Convolutional Networks for Biomedical Relation Extraction (D19-62)
Copied to clipboard
| Challenge: | 80% of the data sets for relation extraction tasks are negative instances, resulting in a lack of syntactic information between two entity mentions. |
| Approach: | They propose a graph convolutional networks model that incorporates dependency parsing and contextualized embedding to capture comprehensive contextual information. |
| Outcome: | The proposed model achieves state-of-the-art F-score on the 2013 drug-drug interaction extraction task. |
A Graph-based Model for Joint Chinese Word Segmentation and Dependency Parsing (2020.tacl-1)
Copied to clipboard
| Challenge: | Chinese word segmentation and dependency parsing suffer from error propagation . a graph-based model can integrate both tasks, but it suffers from performance limitations . |
| Approach: | They propose a graph-based model to integrate Chinese word segmentation and dependency parsing . their model achieves better performance than previous joint models . |
| Outcome: | The proposed model achieves better performance than previous joint models and state-of-the-art results in both Chinese word segmentation and dependency parsing. |
N-LTP: An Open-source Neural Language Technology Platform for Chinese (2021.emnlp-demo)
Copied to clipboard
| Challenge: | Existing tools that teach an independent model for each task are not supported in Chinese. |
| Approach: | They propose an open-source neural language platform supporting six Chinese NLP tasks . source code, documentation, and pre-trained models are available at https://github.com/hit-SCIR/ltp . |
| Outcome: | The proposed platform supports six Chinese NLP tasks. |
COMBO: State-of-the-Art Morphosyntactic Analysis (2021.emnlp-demo)
Copied to clipboard
| Challenge: | COMBO is an end-to-end NLP system for accurate part-of-speech tagging, morphological analysis, and (enhanced) dependency parsing. |
| Approach: | They propose a fully neural NLP system for accurate part-of-speech tagging, morphological analysis, lemmatisation, and (enhanced) dependency parsing. |
| Outcome: | The proposed system predicts categorical morphosyntactic features whilst also exposes their vector representations, extracted from hidden layers. |
Out-of-Domain Discourse Dependency Parsing via Bootstrapping: An Empirical Analysis on Its Effectiveness and Limitation (2022.tacl-1)
Copied to clipboard
| Challenge: | Discourse parsing accuracy degrades significantly on out-of-domain text. |
| Approach: | They propose to use bootstrapping methods to adapt modern discourse dependency parsers to out-of-domain text without additional human supervision. |
| Outcome: | The proposed methods are significantly and consistently effective for unsupervised domain adaptation of discourse dependency parsing, but the low coverage of accurately predicted pseudo labels is a bottleneck for further improvement. |
Trankit: A Light-Weight Transformer-based Toolkit for Multilingual Natural Language Processing (2021.eacl-demos)
Copied to clipboard
| Challenge: | Trankit is a lightweight, pre-trained toolkit for multilingual natural language processing. |
| Approach: | They propose a transformer-based toolkit for multilingual natural language processing that trains pipelines over 100 languages and 90 pretrained pipelines for 56 languages. |
| Outcome: | The proposed tool outperforms existing pipelines over sentence segmentation, part-of-speech tagging, morphological feature tabbing, and dependency parsing while maintaining competitive performance over tokenization, multi-word token expansion, and lemmatization over 90 Universal Dependencies treebanks. |
SanskritShala: A Neural Sanskrit NLP Toolkit with Web-Based Interface for Pedagogical and Annotation Purposes (2023.acl-demo)
Copied to clipboard
| Challenge: | SanskritShala is a neural-based Sanskrit NLP toolkit that is available as a web-based application . |
| Approach: | They propose a neural Sanskrit NLP toolkit that facilitates linguistic analyses for word segmentation, morphological tagging, dependency parsing, and compound type identification. |
| Outcome: | The proposed toolkit reports state-of-the-art performance on benchmark datasets . it is built with easy-to-use interactive data annotation features . |
Evaluating Universal Dependency Parser Recovery of Predicate Argument Structure via CompChain Analysis (2021.starsem-1)
Copied to clipboard
| Challenge: | Compchains are a categorization of the hierarchy of predicate dependency relations present within a UD parse. |
| Approach: | They introduce compchains, a categorization of the hierarchy of predicate dependency relations present within a UD parse. |
| Outcome: | The proposed model performs poorly on sentences with predicate-argument structure with more than one level of embedding. |
VnCoreNLP: A Vietnamese Natural Language Processing Toolkit (N18-5)
Copied to clipboard
| Challenge: | Using word segmenters and POS taggers, Vietnamese NLP pipelines are no longer considered SOTA models for Vietnamese. |
| Approach: | They propose a Java NLP annotation pipeline for Vietnamese that provides rich linguistic annotations. |
| Outcome: | The proposed toolkit provides rich linguistic annotations to facilitate research work on Vietnamese NLP. |
fastHan: A BERT-based Multi-Task Toolkit for Chinese NLP (2021.acl-demo)
Copied to clipboard
| Challenge: | Recently, the need for Chinese natural language processing (NLP) has a dramatic increase for many downstream applications. |
| Approach: | They propose to use Chinese word segmentation (CWS), Part-of-Speech (POS) tagging, named entity recognition (NER), and dependency parsing to train a multi-task model based on a pruned BERT. |
| Outcome: | The proposed model performs better than popular segmentation tools on a non-training corpus. |
DadmaTools: Natural Language Processing Toolkit for Persian Language (2022.naacl-demo)
Copied to clipboard
| Challenge: | Existing tools for Persian language processing are based on conventional non-neural models and do not take full advantage of the latest developments. |
| Approach: | They propose to use a Python neural pipeline for Persian text processing tasks . they use 'parsBERT' to fine-tune the Python pipeline using the PerDT dataset . |
| Outcome: | The proposed toolkit can achieve state-of-the-art performance on multiple NLP tasks. |
Stanza: A Python Natural Language Processing Toolkit for Many Human Languages (2020.acl-demos)
Copied to clipboard
| Challenge: | Existing tools that support only a few major languages are under-optimized for accuracy due to a focus on efficiency or use of less powerful models. |
| Approach: | They introduce a Python natural language processing toolkit that supports 66 languages . they train Stanza on 112 datasets and show it generalizes well on all languages compared to other tools . |
| Outcome: | The proposed toolkit performs well on 112 datasets and is compatible with the popular Java CoreNLP software. |
An In-depth Analysis of the Effect of Lexical Normalization on the Dependency Parsing of Social Media (D19-55)
Copied to clipboard
| Challenge: | Existing natural language processing tools are focused on standard texts, but performance drops when used on a different domain. |
| Approach: | They analyze the effect of manual and automatic lexical normalization for dependency parsing . they conclude that automatic normalization scores close to manually annotated normalization . |
| Outcome: | The proposed approach improves performance on social media data for many tasks . it is unclear which replacements have the most impact and what weaknesses exist in the system . |
A Little Pretraining Goes a Long Way: A Case Study on Dependency Parsing Task for Low-resource Morphologically Rich Languages (2021.eacl-srw)
Copied to clipboard
| Challenge: | Neural dependency parsing has been a success for many domains and languages, but the bottleneck of massive labelled data limits its effectiveness for low resource languages. |
| Approach: | They propose to use morphological knowledge to improve dependency parsing for morphology rich languages in a low-resource setting to perform experiments. |
| Outcome: | The proposed method achieves an average gain of 2 points (UAS) and 3.6 points (LAS) on 10 MRLs in low-resource settings. |
GR-NLP-TOOLKIT: An Open-Source NLP Toolkit for Modern Greek (2025.coling-demos)
Copied to clipboard
Lefteris Loukas, Nikolaos Smyrnioudis, Chrysa Dikonomaki, Spiros Barbakos, Anastasios Toumazatos, John Koutsikakis, Manolis Kyriakakis, Mary Georgiou, Stavros Vassos, John Pavlopoulos, Ion Androutsopoulos
| Challenge: | GR-NLP-TOOLKIT is an open-source natural language processing toolkit for modern Greek. |
| Approach: | They present GR-NLP-TOOLKIT, an open-source natural language processing toolkit for Greek. |
| Outcome: | The toolkit provides state-of-the-art performance in five core NLP tasks . it can be easily installed in Python and is accessible through a demonstration platform on HuggingFace . |
Attention with Dependency Parsing Augmentation for Fine-Grained Attribution (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing fine-grained attribution methods rely on model-internal similarity metrics but lack a fine-grain representation of the data. |
| Approach: | They propose to use model-internal similarity metrics to validate RAG-generated content . they aggregate token-wise evidence through set union operations and integrate dependency parsing to enrich the semantic completeness of target spans. |
| Outcome: | The proposed method outperforms all prior works in the validation of RAG-generated content. |
LINSPECTOR WEB: A Multilingual Probing Suite for Word Representations (D19-3)
Copied to clipboard
| Challenge: | LINSPECTOR WEB is an open source multilingual inspector to analyze word embeddings. |
| Approach: | They propose to use LINSPECTOR WEB to analyze word embeddings in 28 languages. |
| Outcome: | The system performs 16 simple linguistic probing tasks for a diverse set of 28 languages. |
Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLP (2021.eacl-demos)
Copied to clipboard
| Challenge: | Multi-task learning (MTL) has become a standard repertoire in natural language processing (NLP) it enables neural networks to learn tasks in parallel while leveraging the benefits of sharing parameters. |
| Approach: | They propose a toolkit for fine-tuning contextualized embeddings in multi-task settings. |
| Outcome: | The proposed toolkit supports a variety of natural language processing tasks . it enables neural networks to learn tasks in parallel while leveraging the benefits of sharing parameters. |
Neural Dependency Parsing of Biomedical Text: TurkuNLP entry in the CRAFT Structural Annotation Task (D19-57)
Copied to clipboard
| Challenge: | Syntactic analysis (parsing) is a fundamental task in natural language processing (NLP). |
| Approach: | They propose to use the Turku neural parser to adapt it to the biomedical domain . they evaluated custom word embeddings, combination with other in-domain resources . |
| Outcome: | The proposed approach achieved a labeled attachment score of 89.7%, the best among task participants. |
Unleashing the True Potential of Sequence-to-Sequence Models for Sequence Tagging and Structure Parsing (2023.tacl-1)
Copied to clipboard
| Challenge: | Sequence-to-Sequence (S2S) models have been successful on text generation tasks . however, learning complex structures with S2S models remains challenging . |
| Approach: | They propose to use constrained decoding to model part-of-speech tagging, named entity recognition, constituency, and dependency parsing tasks with 3 lexically diverse linearization schemas and corresponding constrained coding methods. |
| Outcome: | The proposed methods outperform the state-of-the-art on four core tasks. |
Using dependency parsing for few-shot learning in distributional semantics (2022.acl-srw)
Copied to clipboard
| Challenge: | Existing methods for few-shot learning use dependency parsing information to learn meaning of rare words based on limited amount of context sentences. |
| Approach: | They propose dependency parsing for few-shot learning to learn meaning of rare words . they use word embedding models as background spaces for few shot learning . |
| Outcome: | The proposed methods enhance the additive baseline model by using dependencies. |
BanSuite: A Unified Toolkit and Software Platform for Low-Resource NLP in Bangla (2026.eacl-demo)
Copied to clipboard
Md. Abu Sayed, Faisal Ahamed Khan, Jannatul Ferdous Tuli, Nabeel Mohammed, Mohammad Ruhul Amin, Mohammad Mamun Or Rashid
| Challenge: | Existing efforts to improve Bangla's NLP performance have focused on isolated tasks such as Part-of-Speech tagging and Named Entity Recognition (NER) but comprehensive, integrated systems for core NLP tasks such Shallow Parsing and Dependency Parser are largely absent. |
| Approach: | They propose to integrate a large-scale, manually annotated Bangla Treebank with high-quality pretrained models for POS tagging, NER, shallow parsing, and dependency parse. |
| Outcome: | The proposed system achieves strong in-domain baseline performance while maintaining high efficiency in resource usage. |
Dependency Parsing with Backtracking using Deep Reinforcement Learning (2022.tacl-1)
Copied to clipboard
| Challenge: | Greedy algorithms for NLP such as transition-based parsing are prone to error propagation. |
| Approach: | They propose to allow transition-based parsing to backtrack in cases where evidence contradicts the current solution. |
| Outcome: | The proposed behavior can be implemented on POS tagging and dependency parsing . it shows that backtracking is an effective means to fight error propagation . |
A structure-enhanced graph convolutional network for sentiment analysis (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work on sentiment analysis and aspect-based sentiment analysis does not exploit syntactic information from dependency parsing. |
| Approach: | They propose a weighted graph convolutional network which exploits syntactic information . they use BERT instead of Bi-LSTM to generate contextualized representations as inputs . |
| Outcome: | The proposed model can exploit rich syntactic information based on feature combination . it can improve on four ABSA tasks out of six and two SA tasks out . |
Linear-Time Modeling of Linguistic Structure: An Order-Theoretic Perspective (2023.emnlp-main)
Copied to clipboard
| Challenge: | a new framework for structured prediction is developed for natural language processing . a systematic approach to structured prediction requires exhaustive pair-wise comparisons of tokens . |
| Approach: | They propose a method that models the relationship between pairs of tokens in a string . they use a parallel method that predicts real numbers for each token in . |
| Outcome: | The proposed method doubles the speed of graph-based dependency parsers and brings 10-times speed-up over graph-driven dependency parses. |
KWJA: A Unified Japanese Analyzer Based on Foundation Models (2023.acl-demo)
Copied to clipboard
Nobuhiro Ueda, Kazumasa Omura, Takashi Kodama, Hirokazu Kiyomaru, Yugo Murawaki, Daisuke Kawahara, Sadao Kurohashi
| Challenge: | KWJA supports a wide range of tasks including typo correction, word segmentation, word normalization, named entity recognition, dependency parsing, PAS analysis, bridging reference resolution, coreference resolution, and discourse relation analysis. |
| Approach: | They propose to build a Japanese text analyzer based on foundation models that performs a wide range of tasks. |
| Outcome: | The proposed model performs better in a multi-task manner than other analyzers with specialized models. |
End-to-End Argument Mining as Biaffine Dependency Parsing (2021.eacl-main)
Copied to clipboard
| Challenge: | Argumentation mining (AM) is a new field of research that uses dependency parsing to analyse arguments. |
| Approach: | They propose a neural end-to-end approach to argument mining based on dependency parsing . their model is biaffine dependency parsed and outperforms the current state-of-the-art . |
| Outcome: | The proposed model outperforms the state-of-the-art in component identification and relation identification. |
Rethinking Self-Attention: Towards Interpretability in Neural Parsing (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work shows that attention mechanisms provide arguably explainable attention distributions that can help to interpret predictions. |
| Approach: | They propose a new self-attention layer where attention heads represent labels. |
| Outcome: | The proposed model obtains state-of-the-art results on the Penn Treebank and Chinese Treebank. |
SHARP: Search-Based Adversarial Attack for Structured Prediction (2022.findings-naacl)
Copied to clipboard
| Challenge: | SHARP is a new attack method for structured prediction models that solves several challenges. |
| Approach: | They propose a black-box adversarial attack method that uses a search-based optimization problem to attack adversarials. |
| Outcome: | The proposed method performs more potent attack than pioneer arts on two structured prediction tasks. |
Dialo-AP: A Dependency Parsing Based Argument Parser for Dialogues (2022.coling-1)
Copied to clipboard
| Challenge: | a recent work on argument mining has focused on parsing monologues, while neglecting dialogues. |
| Approach: | They propose an end-to-end argument parser that constructs argument graphs from dialogues . they use extensive pre-training and curriculum learning to train AM . |
| Outcome: | The proposed system performs all sub-tasks of AM and achieves significant improvements . it is compared to existing systems and validated through human evaluation . |
Viable Dependency Parsing as Sequence Labeling (N19-1)
Copied to clipboard
| Challenge: | Existing work on dependency parsing by sequence labeling suggested that it was impractical. |
| Approach: | They propose to use dependency trees as sequence labels to obtain fast and accurate parsers using a conventional BILSTM-based model. |
| Outcome: | The proposed models are conceptually simple, not needing traditional parsing algorithms or auxiliary structures, and provide a good speed-accuracy tradeoff, with results competitive with more complex approaches. |
Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERT (D19-1)
Copied to clipboard
| Challenge: | Pretrained contextual representation models have pushed forward the state-of-the-art on many NLP tasks. |
| Approach: | They propose to use a model that is pretrained on 104 languages for cross-lingual transfer. |
| Outcome: | The proposed model performs well on 5 NLP tasks covering 39 languages from various language families. |
Neural Syntactic Generative Models with Exact Marginalization (N18-1)
Copied to clipboard
| Challenge: | Recent models have added structure to recurrent neural networks at the cost of giving up exact inference, or using soft structure instead of latent variables. |
| Approach: | They propose a syntactic generative model with exact marginalization that supports dependency parsing and language modeling. |
| Outcome: | The proposed models achieve state-of-the-art for supervised dependency parsing and language modeling. |
Improving Code-Switching Dependency Parsing with Semi-Supervised Auxiliary Tasks (2022.findings-naacl)
Copied to clipboard
| Challenge: | Code-switching dependency parsing is a challenging task due to the scarcity of necessary resources and structural difficulties embedded in code-switch languages. |
| Approach: | They propose to use sequence labeling models as auxiliary tasks for code-switched dependency parsing in a semi-supervised scheme and acquire state-of-the-art scores on all studied languages. |
| Outcome: | The proposed model outperforms the previous model by 7.4 LAS points on average on all of the studied languages. |
Robust Multilingual Part-of-Speech Tagging via Adversarial Training (N18-1)
Copied to clipboard
| Challenge: | Adversarial training (AT) is a powerful regularization method for neural networks, aiming to achieve robustness to input perturbations. |
| Approach: | They propose and analyze a neural POS tagging model that exploits adversarial training by training on unmodified and adversarials. |
| Outcome: | The proposed model improves overall tagging accuracy and prevents over-fitting in low resource languages and boosts tabbing accuracy for rare / unseen words. |
Instance-Based Neural Dependency Parsing (2021.tacl-1)
Copied to clipboard
Hiroki Ouchi, Jun Suzuki, Sosuke Kobayashi, Sho Yokoi, Tatsuki Kuribayashi, Masashi Yoshikawa, Kentaro Inui
| Challenge: | Existing models that use instance-based inference for dependency parsing are difficult to understand for humans. |
| Approach: | They develop neural models that adopt an interpretable inference process for dependency parsing. |
| Outcome: | The proposed models achieve competitive accuracy with standard neural models and have plausibility of instance-based explanations. |
Universal Dependency Parsing for Hindi-English Code-Switching (N18-1)
Copied to clipboard
| Challenge: | Code-switching data often need additional processes such as language identification, normalization and/or back-transliteration to be processed. |
| Approach: | They propose a neural stacking model that leverages part-of-speech tags and syntactic tree annotations in tweets to parse code-switching data. |
| Outcome: | The proposed model is 1.5% better than the augmented model and 3.8% better than one which uses first-best normalization and/or back-transliteration. |
Morphology-Aware Multi-Granularity Representation Learning for Agglutinative Languages (2026.acl-srw)
Copied to clipboard
| Challenge: | Existing methods for learning low-resource agglutinative languages are limited to word and phrase levels. |
| Approach: | They propose a morphology-aware gated multi-granularity pre-training framework for agglutinative languages . framework leverages morphological knowledge and integrates a word-level encoder to capture contextual semantics . |
| Outcome: | The proposed framework improves on Mongolian and Turkish agglutinative languages . it leverages morphological knowledge and integrates tagging and segmentation to build fine-grained representations . |
Semi-Supervised Semantic Role Labeling with Cross-View Training (D19-1)
Copied to clipboard
| Challenge: | Recent approaches rely on expensive annotations and are unavailable in low resource scenarios (e.g., rare languages or domains). |
| Approach: | They propose an end-to-end SRL model which leverages unlabeled data and propose to reduce the annotation effort involved via semi-supervised learning. |
| Outcome: | The proposed model outperforms the state-of-the-art in English and consistently improves performance in other languages, including Chinese, German, and Spanish. |
A Systematic Study of Leveraging Subword Information for Learning Word Representations (N19-1)
Copied to clipboard
| Challenge: | Existing word representation models for morphologically rich languages use subword-level information, but their systematic comparative analysis across typologically diverse languages and tasks is still missing. |
| Approach: | They propose a framework for learning subword-informed word representations that allows for easy experimentation with different segmentation and composition components. |
| Outcome: | The proposed framework allows for easy experimentation with different segmentation and composition components, as well as advanced techniques based on position embeddings and self-attention. |
On Finding the K-best Non-projective Dependency Trees (2021.acl-long)
Copied to clipboard
| Challenge: | Existing work on finding the one-best dependency tree has not extended this to finding the K-best tree. |
| Approach: | They propose to simplify the K-best spanning tree algorithm by decoding the K best dependency trees with a root constraint. |
| Outcome: | The proposed algorithm can be used to find the K-best dependency trees without root constraint. |
Building a User-Generated Content North-African Arabizi Treebank: Tackling Hell (2020.acl-main)
Copied to clipboard
Djamé Seddah, Farah Essaidi, Amal Fethi, Matthieu Futeral, Benjamin Muller, Pedro Javier Ortiz Suárez, Benoît Sagot, Abhishek Srivastava
| Challenge: | a treebank for a north-African Arabic dialect known for code-switching is made freely available . authors: geopolitical events are a factor highlighting a language deficiency in terms of natural language processing resources . |
| Approach: | They propose to make a treebank for a romanized user-generated content variety of Algerian . they supplement it with 50k unlabeled sentences from common crawl and web-crawled data . |
| Outcome: | The proposed treebank is made of 1500 sentences, fully annotated in morpho-syntax and universal dependency syntax, with full translation at both the word and sentence levels. |
Assessing Emoji Use in Modern Text Processing Tools (2021.acl-long)
Copied to clipboard
| Challenge: | Emojis are textual elements that are encoded as characters but rendered as small digital images or icons that can be used to express an idea or emotion. |
| Approach: | They propose to use a set of popular NLP tools to assess the support of emojis in tweets. |
| Outcome: | The proposed methods show that many systems still have notable shortcomings when operating on text containing emojis. |
Combining (Second-Order) Graph-Based and Headed-Span-Based Projective Dependency Parsing (2022.findings-acl)
Copied to clipboard
| Challenge: | Existing graph-based methods that score dependency trees do not score dependency arcs at all. |
| Approach: | They propose a headed-span-based method that decomposes the score of a dependency tree into scores of headed spans. |
| Outcome: | The proposed method improves over first-order graph-based methods, but does not score dependency arcs at all. |
How much pretraining data do language models need to learn syntax? (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pretraining methods are convenient, but expensive in terms of time and resources. |
| Approach: | They investigate the impact of pretraining data size on the syntactic capabilities of RoBERTa by using syntaktic structural probes to determine whether models pretrained on more data encode a higher amount of syntastic information. |
| Outcome: | The proposed models perform better on part-of-speech tagging, dependency parsing and paraphrase identification. |
Parsing with Multilingual BERT, a Small Corpus, and a Small Treebank (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Pretrained multilingual contextual representations have shown great success, but their benefits do not apply equally to all language varieties. |
| Approach: | They propose to use additional language-specific pretraining and vocabulary augmentation to adapt multilingual models to low-resource settings. |
| Outcome: | The proposed methods significantly improve performance over baselines, especially in the lowest-resource cases, and demonstrate the importance of the relationship between such models’ pretraining data and target language varieties. |
BERTrade: Using Contextual Embeddings to Parse Old French (2022.lrec-1)
Copied to clipboard
| Challenge: | a growing interest in digital humanities for automatic processing and annotation of historical texts is generating new models for historical languages. |
| Approach: | They use POS-tagging and dependency parsing to evaluate contextual word embedding models . Old French is one of the historical languages for which they have the largest amount of syntactically annotated data . |
| Outcome: | The proposed model can be used to improve performance in Old French, the authors show . they use POS-tagging and dependency parsing to evaluate the model's quality . |
K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters (2021.findings-acl)
Copied to clipboard
Ruize Wang, Duyu Tang, Nan Duan, Zhongyu Wei, Xuanjing Huang, Jianshu Ji, Guihong Cao, Daxin Jiang, Ming Zhou
| Challenge: | Existing methods for injecting knowledge into pre-trained models are inconsistent and can flush out knowledge when multiple kinds of knowledge are injected. |
| Approach: | They propose a framework that retains the original parameters of pre-trained models fixed and supports the development of versatile knowledge-infused models. |
| Outcome: | The proposed framework retains the original parameters of the pre-trained model fixed and supports the development of versatile knowledge-infused models. |
Hexatagging: Projective Dependency Parsing as Tagging (2023.acl-short)
Copied to clipboard
| Challenge: | Using a pretrained language model, we can train language models on increasingly large amounts of data. |
| Approach: | They propose a dependency parser that constructs dependency trees by tagging words with elements from a finite set of possible tags. |
| Outcome: | The proposed approach achieves state-of-the-art performance of 96.4 LAS and 97.4 UAS on the Penn Treebank test set. |
“Wikily” Supervised Neural Translation Tailored to Cross-Lingual Tasks (2021.emnlp-main)
Copied to clipboard
| Challenge: | Unsupervised neural machine translation models perform well in low-resource or distant languages. |
| Approach: | They propose a model that leverages Wikipedia for machine translation and cross-lingual tasks without supervision from external parallel data or supervised models in target language. |
| Outcome: | The proposed model outperforms supervised models in Arabic and English translation tasks. |
Stack-Pointer Networks for Dependency Parsing (P18-1)
Copied to clipboard
| Challenge: | Existing approaches to dependency parsing are local and greedy transitionbased . StackPtr parsers use the information of whole sentences and previously derived subtree structures . |
| Approach: | They propose a stack-pointer network-based dependency parser that reads whole sentence and builds dependency tree top-down in a depth-first fashion. |
| Outcome: | The proposed model reads and encodes whole sentence, then builds dependency tree top-down (from root-to-leaf) in a depth-first fashion. |
Twitter Universal Dependency Parsing for African-American and Mainstream American English (P18-1)
Copied to clipboard
| Challenge: | We analyze the performance disparities between AAE and Mainstream American English (MAE) because of Twitter-specific conventions and dialectal language. |
| Approach: | They develop a dataset of 500 tweets, 250 of which are in AAE, within the Universal Dependencies 2.0 framework and annotate it. |
| Outcome: | The proposed model improves performance for AAE tweets with no or very little in-domain labeled data and assesses its lexical and syntactic features. |
Extracting Victim Counts from Text (2023.eacl-main)
Copied to clipboard
| Challenge: | Using tagging and regex methods, data on injured, displaced, or abused victims is difficult . data on earthquake injuries and deaths is scarce, subjective, or biased . |
| Approach: | They compare tagging approaches to extract injured, displaced, or abused victims . they discuss calibration and investigate out-of-distribution and few-shot performance . |
| Outcome: | The proposed model is among the first to apply numeracy-focused large language models in a real-world use case with a positive impact. |
Dependency parsing with structure preserving embeddings (2021.eacl-main)
Copied to clipboard
| Challenge: | Modern neural approaches to dependency parsing are trained to predict a tree structure by learning a contextual representation for tokens in a sentence and a head–dependent scoring function. |
| Approach: | They propose to combine a contextual representation for tokens and a head–dependent scoring function to learn interpretable representations by training a parser to explicitly preserve structural properties of a tree. |
| Outcome: | The proposed approach yields strong tree distance preservation and parsing performance on par with a competitive graph-based parser. |
GraphDialog: Integrating Graph Knowledge into End-to-End Task-Oriented Dialogue Systems (2020.emnlp-main)
Copied to clipboard
| Challenge: | End-to-end task-oriented dialogue systems aim to generate system responses directly from plain text inputs. |
| Approach: | They propose a recurrent cell architecture which exploits the structural information in dialogue history . they propose recursive cell architecture to allow representation learning on graphs . |
| Outcome: | The proposed model improves on two different datasets on task-oriented dialogues. |
Unsupervised Cross-Lingual Transfer of Structured Predictors without Source Data (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent successes of NLP systems require large amounts of labelled data for structured prediction tasks. |
| Approach: | They propose a method for unsupervised transfer from multiple input models for structured prediction using a cross-lingual setup. |
| Outcome: | The proposed method produces less noisy labels for the distant supervision. |
APGN: Adversarial and Parameter Generation Networks for Multi-Source Cross-Domain Dependency Parsing (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models for dependency parsing use labeled training data for several fixed domains, but performance drops when labeles only exist for several out-domains. |
| Approach: | They propose a model for multi-source cross-domain dependency parsing that uses a parameter generation network and adversarial network for learning domain-invariant representations. |
| Outcome: | The proposed model improves cross-domain parsing performance by about 2 points over strong BERT-enhanced baselines over a recently released dataset for multi-domain dependency parse. |
Everything Is All It Takes: A Multipronged Strategy for Zero-Shot Cross-Lingual Information Extraction (2021.emnlp-main)
Copied to clipboard
Mahsa Yarmohammadi, Shijie Wu, Marc Marone, Haoran Xu, Seth Ebner, Guanghui Qin, Yunmo Chen, Jialiang Guo, Craig Harman, Kenton Murray, Aaron Steven White, Mark Dredze, Benjamin Van Durme
| Challenge: | Zero-shot cross-lingual information extraction (IE) is a technique for training data in a source language but not in . |
| Approach: | They explore techniques including data projection and self-training to improve zero-shot cross-lingual information extraction (IE) IE is a construction of an IE model for some target language given existing annotations exclusively in English. |
| Outcome: | The proposed techniques show that they perform better than any single strategy. |
Frustratingly Easy Performance Improvements for Low-resource Setups: A Tale on BERT and Segment Embeddings (2022.lrec-1)
Copied to clipboard
| Challenge: | Understanding why contextualized embeddings work is still an active area of research. |
| Approach: | They propose to use a BERT architecture to encode a sub-word, position and a segment embedding as input representations for each sub- word. |
| Outcome: | The proposed model performs well on single-sentence prediction tasks while swapping segment IDs in paired-sentent tasks. |
An Empirical Exploration of Local Ordering Pre-training for Structured Prediction (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Recent studies have shown that pre-training contextualized encoders with language model objectives is effective for structured prediction. |
| Approach: | They propose a semi-supervised method for pre-training contextualized encoders with language model objectives. |
| Outcome: | The proposed method is effective on three typical structured prediction tasks in four languages. |
Dependency Parsing for Spoken Dialog Systems (D19-1)
Copied to clipboard
| Challenge: | Dependency parsing of conversational input can help to understand dialogs . currently available annotation schemes do not adapt well to spoken human-machine dialogs. |
| Approach: | They propose an annotation scheme that extends Universal Dependencies guidelines to spoken dialogs. |
| Outcome: | The proposed scheme disambiguates relationships between entities extracted from dialogs . it is better than existing models on public datasets and fine-tuned on ConvBank data . |
Unsupervised Korean Word Sense Disambiguation using CoreNet (L18-1)
Copied to clipboard
| Challenge: | Unsupervised learning based Korean word sense disambiguation is needed to distinguish between sense candidates. |
| Approach: | They investigated unsupervised Korean word sense disambiguation using CoreNet, a Korean lexical semantic network. |
| Outcome: | The proposed method exhibited an 80.9% accuracy on the datasets constructed and proved to be effective for practical applications. |
Automatic Extraction of Nominal Phrases from German Learner Texts of Different Proficiency Levels (2024.lrec-main)
Copied to clipboard
| Challenge: | a pilot study has found that inflecting determiners and adjectives correctly is a challenge for learners of German. |
| Approach: | They propose to use dependency parsing to extract nouns, grammatical heads and dependents that have to agree with the noun in German. |
| Outcome: | The proposed method performs well on CEFR levels A1-B1 but not level B2 texts. |
Dependency Parsing as MRC-based Span-Span Prediction (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for dependency parsing address the issue that edges should be constructed at the text span/subtree level rather than word level. |
| Approach: | They propose a method that constructs dependency trees by directly modeling span-span relations by modeling subtree-subtree relationships. |
| Outcome: | The proposed method constructs dependency trees by modeling span-span relations . it can retrieve missing spans in the span proposal stage, which leads to higher recall . |
Adversarial Attack and Defense of Structured Prediction Models (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to building effective adversarial attackers focus on classification problems. |
| Approach: | They propose a framework that learns to attack a structured prediction model with feedbacks from multiple reference models. |
| Outcome: | The proposed framework is able to attack state-of-the-art models and boost them with training . it is based on a sequence-to-sequence model with feedbacks from multiple reference models . |
Unsupervised Domain Adaptation Method with Semantic-Structural Alignment for Dependency Parsing (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for dependency parsing are often of the pseudo-annotation type, but they fail to consider the change of model structure for domain adaptation. |
| Approach: | They propose a method that accomplishes unsupervised cross-domain dependency parsing without using labeled data. |
| Outcome: | The proposed method achieves consistent performance improvement on CODT1 and CTB9 domains. |
On the Sensitivity and Stability of Model Interpretations in NLP (2022.acl-long)
Copied to clipboard
| Challenge: | Recent years have witnessed the emergence of post-hoc interpretations that aim to uncover how NLP models make predictions. |
| Approach: | They propose two new criteria that provide complementary notions of faithfulness to removal-based criteria. |
| Outcome: | The proposed methods overcome limitations of gradient-based methods on removal-based criteria and overcome limitations in the proposed methods. |
Reinforced Training Data Selection for Domain Adaptation (P19-1)
Copied to clipboard
| Challenge: | Existing approaches to learn domains with massive data are not easy to implement and require a predefined threshold. |
| Approach: | They propose a framework that searches for training instances relevant to the target domain and learns better representations for them. |
| Outcome: | The proposed framework is effective in data selection and representation, but generalized to accommodate different NLP tasks. |
ATP: AMRize Then Parse! Enhancing AMR Parsing with PseudoAMRs (2022.findings-naacl)
Copied to clipboard
| Challenge: | Abstract Meaning Representation (AMR) implicitly involves compound semantic annotations. |
| Approach: | They propose to use auxiliary tasks which are semantically or formally related to enhance AMR parsing. |
| Outcome: | The proposed method achieves state-of-the-art performance on benchmarks especially in topology-related scores. |
Generalizing Natural Language Analysis through Span-relation Representations (2020.acl-main)
Copied to clipboard
| Challenge: | a large number of natural language processing tasks are generated with specially designed architectures. |
| Approach: | They propose to represent a wide variety of tasks in a single unified format . they perform extensive experiments to demonstrate benefits of multi-task learning . |
| Outcome: | The proposed model performs comparable to state-of-the-art models on 10 tasks . it also shows that it can analyze differences and similarities in how the model handles different tasks compared to other models . |
Semi-Supervised Sequence Modeling with Cross-View Training (D18-1)
Copied to clipboard
| Challenge: | Unsupervised representation learning algorithms such as word2vec and ELMo only learn from task-specific labeled data during the main training phase. |
| Approach: | They propose a semi-supervised learning algorithm that improves the representations of a Bi-LSTM sentence encoder using a mix of labeled and unlabeled data. |
| Outcome: | The proposed algorithm improves the representations of a Bi-LSTM sentence encoder using a mix of labeled and unlabeled data. |
Semi-Supervised Dependency Parsing with Arc-Factored Variational Autoencoding (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for dependency parsing use unlabelled data to compensate for the lack of training corpora. |
| Approach: | They propose semi-supervised dependency parsing methods that utilize unlabelled data to compensate for the scarcity of labelled training corpora. |
| Outcome: | The proposed model overcomes the tree constraint and the complexity of the training procedure while avoiding the challenges brought by the tree constraints. |
Head-Driven Phrase Structure Grammar Parsing on Penn Treebank (P19-1)
Copied to clipboard
| Challenge: | Head-driven phrase structure grammars have a uniform formalism representing rich contextual syntactic and even semantic meanings. |
| Approach: | They propose to integrate constituent and dependency formal representations into head-driven phrase structure. |
| Outcome: | The proposed parser achieves state-of-the-art performance on Penn Treebank and Chinese Penn TreeBank. |
Revisiting the Effects of Leakage on Dependency Parsing (2022.findings-acl)
Copied to clipboard
| Challenge: | Recent work shows that treebank size and linguistic variation are important factors that explain the variation in dependency parsing performance. |
| Approach: | They propose a measure of leakage that explains and correlates with observed performance variation. |
| Outcome: | The proposed measure explains and correlates with observed performance variation. |
When Only Time Will Tell: Interpreting How Transformers Process Local Ambiguities Through the Lens of Restart-Incrementality (2024.acl-long)
Copied to clipboard
| Challenge: | In incremental models, one interpretation is possible, but models that can revise can do so if the ambiguity is resolved. |
| Approach: | They propose an interpretable way to analyse incremental states in a bidirectional way . they propose to use a model that can update internal states to reflect the garden path effect . |
| Outcome: | The proposed model shows that it can perform revisions and recover if the label is incorrect. |
When Beards Start Shaving Men: A Subject-object Resolution Test Suite for Morpho-syntactic and Semantic Model Introspection (2020.coling-main)
Copied to clipboard
| Challenge: | In German, subject-object resolution is the second most error-prone task in syntactic parsing, after prepositional phrase attachment. |
| Approach: | They introduce the SORTS Subject-Object Resolution Test Suite of German minimal sentence pairs for model introspection. |
| Outcome: | The proposed test suite is based on the annotations of 8,502 transitive clauses with 8 word order patterns, 5 morphological and syntactic and 11 semantic property classes. |
MRL Parsing Without Tears: The Case of Hebrew (2024.findings-acl)
Copied to clipboard
| Challenge: | a new approach to parsing morphologically rich languages (MRLs) is needed to overcome the deficiencies of current approaches. |
| Approach: | They propose a "flipped pipeline" where multiple layers are predicted independently on whole-token basis and then synthesized. |
| Outcome: | The proposed model achieves near-SOTA performance on Hebrew NLP tasks. |
Building an Ellipsis-aware Chinese Dependency Treebank for Web Text (L18-1)
Copied to clipboard
| Challenge: | ellipsis is a common linguistic phenomenon that some words are left out as they are understood from the context, especially in oral utterance. |
| Approach: | They propose to use a Chinese dependency treebank to facilitate the parsing of web text . they propose to restore omissions and reserve contexts in the web text to improve dependency parsers . |
| Outcome: | The proposed framework enables the parsing of web text from online microblogs. |
VICTR: Visual Information Captured Text Representation for Text-to-Vision Multimodal Tasks (2020.coling-main)
Copied to clipboard
| Challenge: | Existing text-to-image generation models focus on generating high resolution images and neglect understanding text descriptions. |
| Approach: | They propose a visual contextual text representation which captures rich visual semantic information of objects from text input. |
| Outcome: | The proposed visual contextual text representation improves on the state-of-the-art models. |
Adaptation of Deep Bidirectional Transformers for Afrikaans Language (2020.lrec-1)
Copied to clipboard
| Challenge: | Pretrained language models can be trained in unsupervised manner, but can be difficult to implement because of the amount of data and computational resources needed for pretraining. |
| Approach: | They propose a model for Afrikaans based on bidirectional encoder representation from transformers. |
| Outcome: | The proposed model outperforms the existing models in part-of-speech tagging, named-entity recognition, and dependency parsing tasks. |
Towards Protecting Vital Healthcare Programs by Extracting Actionable Knowledge from Policy (2021.findings-acl)
Copied to clipboard
Vanessa Lopez, Nagesh Yadav, Gabriele Picco, Inge Vejsbjerg, Eoin Carrol, Seamus Brady, Marco Luca Sbodio, Lam Thanh Hoang, Miao Wei, John Segrave
| Challenge: | In the U.S., an estimated annual amount of USD$20-30B is lost to Fraud, Waste and abuse (FWA) |
| Approach: | They propose a method for automatically extracting knowledge from healthcare policy documents into a semantically-meaningful knowledge graph of rules. |
| Outcome: | The proposed method fuses advances in dependency parsing with a policy ontology to transform the content of regulatory healthcare policy into human-friendly policy rules with human oversight. |
Improved Dependency Parsing using Implicit Word Connections Learned from Unlabeled Data (D18-1)
Copied to clipboard
| Challenge: | Pre-trained word embeddings and language models cannot capture word connections in a sentence. |
| Approach: | They propose to implicitly capture word connections from unlabeled data by word ordering model with self-attention mechanism. |
| Outcome: | The proposed model achieves 96.35% UAS and 95.25% LAS on the English PTB dataset. |
Cross-Lingual Syntactic Transfer through Unsupervised Adaptation of Invertible Projections (P19-1)
Copied to clipboard
| Challenge: | Current systems for syntactic analysis tasks rely heavily on large scale annotated data. |
| Approach: | They propose to learn a generative model with a structured prior that uses labeled source and unlabeled target data jointly. |
| Outcome: | The proposed model improves on part-of-speech tagging and dependency parsing tasks on English as the only source corpus and on a wide range of target languages. |
A Framework for Understanding the Role of Morphology in Universal Dependency Parsing (D18-1)
Copied to clipboard
| Challenge: | a measure of morphological complexity is used to characterize syntactic information in word embeddings. |
| Approach: | They propose a measure of morphological complexity in terms of governor-dependent preferential attachment that explains parsing performance. |
| Outcome: | The proposed framework improves parsing performance on morphologically rich languages using morphology as a syntactic marker. |
Data-Efficient French Language Modeling with CamemBERTa (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in NLP have significantly improved the performance of language models on a variety of tasks. |
| Approach: | They introduce a French DeBERTa model that builds upon the DeBERTAV3 architecture and training objective and evaluate its performance on a variety of French downstream tasks and datasets. |
| Outcome: | The proposed model outperforms BERT-based models on most tasks given the same amount of training tokens and trained on 30% of its input tokens. |
Enhancing Aspect-level Sentiment Analysis with Word Dependencies (2021.eacl-main)
Copied to clipboard
| Challenge: | Existing approaches to enhance aspect-level sentiment analysis have omitted syntactic information . experimental results show that our approach outperforms baseline models on all datasets . |
| Approach: | They propose to leverage word dependencies to enhance aspect-level sentiment analysis . they propose to use key-value memory networks to leverage different dependency results . |
| Outcome: | The proposed approach outperforms baseline models on all datasets and achieves state-of-the-art performance on three of them. |
A Unifying Theory of Transition-based and Sequence Labeling Parsing (2020.coling-main)
Copied to clipboard
| Challenge: | Existing parsers that read sentences from left to right are not learning to parse them. |
| Approach: | They propose a mapping from transition-based parsing algorithms that read sentences from left to right to sequence labeling encodings of syntactic trees. |
| Outcome: | The proposed algorithms are learnable and comparable to existing encodings. |
Women’s Syntactic Resilience and Men’s Grammatical Luck: Gender-Bias in Part-of-Speech Tagging and Dependency Parsing (P19-1)
Copied to clipboard
| Challenge: | linguistic studies have shown the prevalence of various lexical and grammatical patterns in texts authored by a person of a particular gender, but models for part-of-speech tagging and dependency parsing have not adapted to account for these differences. |
| Approach: | They annotate the Wall Street Journal part of the Penn Treebank with the gender information of the articles’ authors and build taggers and parsers trained on this data. |
| Outcome: | The proposed model can account for gendered differences in syntactic tasks and highlight future venues for developing more accurate taggers and parsers. |
Improving Unsupervised Question Answering via Summarization-Informed Question Generation (2021.emnlp-main)
Copied to clipboard
| Challenge: | Question Generation (QG) is the production of meaningful questions given a set of input passages and corresponding answers. |
| Approach: | They propose a method which uses questions generated heuristically from news summaries as a source of training data for a QG system. |
| Outcome: | The proposed method outperforms previous unsupervised models on three in-domain datasets and three out-of-domain ones. |
BKTreebank: Building a Vietnamese Dependency Treebank (L18-1)
Copied to clipboard
| Challenge: | In this paper, we present the building of a dependency treebank for Vietnamese . |
| Approach: | They propose to build a Vietnamese dependency treebank using automatic taggers and automatic tagging. |
| Outcome: | The proposed treebank is a useful resource for Vietnamese language processing. |
Moving TIGER beyond Sentence-Level (L18-1)
Copied to clipboard
| Challenge: | TIGER 2.2-doc is a new set of annotations for the German TIger corpus. |
| Approach: | They propose a new set of annotations for the German TIGER corpus . they introduce new document-level annotations: authors and their gender. |
| Outcome: | The new annotations improve the TIGER corpus and its structure and authors and gender. |
Multilingual Dependency Parsing for Low-Resource Languages: Case Studies on North Saami and Komi-Zyrian (L18-1)
Copied to clipboard
| Challenge: | Developing systems for low-resource languages is a crucial issue for Natural Language Processing (NLP). |
| Approach: | They propose a method for parsing low-resource languages with very small training corpora using multilingual word embeddings and annotated corporata of larger languages. |
| Outcome: | The proposed method improves dependency parsing for low-resource languages with very small training corpora compared to previous work . it also explores whether contemporary contact languages or genetically related languages would be the most fruitful starting point for multilingual parsers. |
ScriptMix: Mixing Scripts for Low-resource Language Parsing (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing work has considered transliteration and vocabulary augmentation, but the consideration of combining the two has been lacking. |
| Approach: | They propose a multilingual pretrained language model that combines two strengths and overcomes the hurdle of combining them. |
| Outcome: | The proposed model improves POS accuracy by 14% and improves DEP LAS score by 5.6%. |
A Novel Multi-Task Learning Approach for Context-Sensitive Compound Type Identification in Sanskrit (2022.coling-1)
Copied to clipboard
Jivnesh Sandhan, Ashish Gupta, Hrishikesh Terdalkar, Tushar Sandhan, Suvendu Samanta, Laxmidhar Behera, Pawan Goyal
| Challenge: | Previously, compounding is a problem of identifying semantic relations between components of a word. |
| Approach: | They propose a multi-task learning architecture which incorporates contextual information and enriches syntactic information using morphological tagging and dependency parsing as auxiliary tasks. |
| Outcome: | The proposed architecture shows 6.1 points accuracy and 7.7 points (F1-score) absolute gain in English and Marathi languages. |
From Zero to Hero: On the Limitations of Zero-Shot Language Transfer with Multilingual Transformers (2020.emnlp-main)
Copied to clipboard
| Challenge: | Existing studies show that multilingual transformers are less effective in resource-lean scenarios and for distant languages. |
| Approach: | They propose to use massively multilingual transformers to pretrain languages . they show that MMTs are less effective in resource-lean scenarios and distant languages if they are pre-trained via language modeling . |
| Outcome: | The proposed model is less effective in resource-lean scenarios and for distant languages than cross-lingual word embeddings. |
SciBERT: A Pretrained Language Model for Scientific Text (D19-1)
Copied to clipboard
| Challenge: | SciBERT is a pretrained language model based on BERT to improve performance on scientific NLP tasks. |
| Approach: | They propose a pretrained language model based on BERT to improve NLP performance . they evaluate on sequence tagging, sentence classification and dependency parsing . |
| Outcome: | The proposed model improves on sequence tagging, sentence classification and dependency parsing tasks with datasets from a variety of scientific domains. |
Systematic Inequalities in Language Technology Performance across the World’s Languages (2022.acl-long)
Copied to clipboard
| Challenge: | Recent studies have revealed that NLP is limited to a subset of the world’s 6,500 languages. |
| Approach: | They propose a framework for estimating the global utility of language technologies as revealed in a comprehensive snapshot of recent publications in NLP. |
| Outcome: | The proposed framework estimates the global utility of language technologies as revealed in a comprehensive snapshot of recent publications in NLP. |
Neural Reranking for Dependency Parsing: An Evaluation (2020.acl-main)
Copied to clipboard
| Challenge: | Recent work shows that neural rerankers can improve dependency parsing results over the top k trees produced by a base parser. |
| Approach: | They propose to use a discriminative reranker to improve dependency parsing results . they propose to incorporate global information into the model to improve parse accuracies . |
| Outcome: | The proposed model outperforms existing models on English and German and Czech, and is the only one to improve on German and Chinese data. |
A Search-based Neural Model for Biomedical Nested and Overlapping Event Detection (D19-1)
Copied to clipboard
| Challenge: | Existing structured prediction tasks target nested and overlapping events . a new structured prediction model is proposed that uses a relation graph to detect overlapping and nesting events. |
| Approach: | They propose a search-based neural network structured prediction model that treats the task as a searching problem on a relation graph of trigger-argument structures. |
| Outcome: | The proposed model performs comparable to the state-of-the-art model Turku Event Extraction System (TEES) on the BioNLP Cancer Genetics (CG) Shared Task 2013 without the use of syntactic and hand-engineered features. |
Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERT (2020.acl-main)
Copied to clipboard
| Challenge: | Recent pre-trained language models achieve state-of-the-art performance for downstream NLP tasks. |
| Approach: | They propose a parameter-free probing technique for analyzing pre-trained language models . their method does not require direct supervision from probing tasks . |
| Outcome: | The proposed method improves on linguistically-uninformed baselines on pre-trained language models. |
Low-Resource Syntactic Transfer with Unsupervised Source Reordering (N19-1)
Copied to clipboard
| Challenge: | Existing methods for dependency parsing use word order differences between source and target languages. |
| Approach: | They propose a cross-lingual transfer method that takes into account word order differences between source and target languages. |
| Outcome: | The proposed method improves on 68 treebanks (38 languages) on a target language. |
Keep it Surprisingly Simple: A Simple First Order Graph Based Parsing Model for Joint Morphosyntactic Parsing in Sanskrit (2020.emnlp-main)
Copied to clipboard
| Challenge: | Morphologically rich languages benefit from joint processing of morphology and syntax, as compared to pipeline architectures. |
| Approach: | They propose a graph-based model for joint morphological parsing and dependency parser in Sanskrit using the Energy based model framework. |
| Outcome: | The proposed model outperforms standalone morphological parsers in morphology and syntax parsing, and in dependency parser. |
Please Mind the Root: Decoding Arborescences for Dependency Parsing (2020.emnlp-main)
Copied to clipboard
| Challenge: | a dependency tree has a root constraint, but only one edge may emanate from the root node. |
| Approach: | They propose an algorithm which enforces a root constraint without compromising the original runtime. |
| Outcome: | The proposed algorithm satisfies the constraint without compromising the original runtime. |
Polyglot Contextual Representations Improve Crosslingual Transfer (N19-1)
Copied to clipboard
| Challenge: | Existing methods for crosslingual transfer use multilingual word embeddings, but contextual word representations are not yet available. |
| Approach: | They propose a method to produce multilingual contextual word representations by training a single language model on text from multiple languages. |
| Outcome: | The proposed method compares model models to monolingual and non-contextual variants and shows that polyglot learning can be beneficial for multilingual representations. |
Typological Features for Multilingual Delexicalised Dependency Parsing (N19-1)
Copied to clipboard
| Challenge: | Existing universal models to describe the syntax of languages are debated for decades . a new study examines the plausibility of universal grammars in dependency parsing . |
| Approach: | They propose to use typological features to describe the syntax of languages to train a multilingual dependency parser. |
| Outcome: | The proposed model can be trained on 40 languages with the help of typological features. |
OFrLex: A Computational Morphological and Syntactic Lexicon for Old French (2020.lrec-1)
Copied to clipboard
| Challenge: | Using heterogeneous language resources, we extract structured and exploitable information from a large-coverage morphological and syntactic Old French lexicon. |
| Approach: | They propose to use a large-coverage morphological and syntactic Old French lexicon to extract structured and exploitable information from heterogeneous language resources. |
| Outcome: | The proposed extension technique will be validated manually in the near future and take advantage of OFrLex’s viewing, searching and editing interface. |
Construction of Large-scale English Verbal Multiword Expression Annotated Corpus (L18-1)
Copied to clipboard
| Challenge: | In this paper, we focus on verbal MWEs, whose accurate recognition is challenging because they could be discontinuous. |
| Approach: | They conduct large-scale annotations of VMWEs on the Wall Street Journal portion of Ontonotes . they first construct a VMwe dictionary based on the english-language Wiktionary . |
| Outcome: | The proposed resource annotates 7,833 VMWE instances belonging to various categories . the authors hope the results will help to develop models for MWE recognition and dependency parsing . |
MAD-G: Multilingual Adapter Generation for Efficient Cross-Lingual Transfer (2021.findings-emnlp)
Copied to clipboard
Alan Ansell, Edoardo Maria Ponti, Jonas Pfeiffer, Sebastian Ruder, Goran Glavaš, Ivan Vulić, Anna Korhonen
| Challenge: | Massively multilingual transformers (MMTs) have benefited from additional training of language-specific adapters, but this approach is not viable for the vast majority of languages due to limitations in their corpus size or compute budgets. |
| Approach: | They propose a multilingual ADapter generation approach which contextually generates language adapters from language representations based on typological features. |
| Outcome: | The proposed method improves cross-lingual transfer performance on part-of-speech tagging, dependency parsing, and named entity recognition tasks while remaining cost-effective. |
How does the brain process syntactic structure while listening? (2023.findings-acl)
Copied to clipboard
| Challenge: | Syntactic parsing is the task of assigning a syntactical structure to a sentence. |
| Approach: | They investigate the predictive power of the brain encoding models in three settings: individual performance of constituency and dependency parsing based embedding methods, relative effectiveness of each of the syntactic parsers, and relative importance of syntaktic information versus semantic information using BERT embeddngs. |
| Outcome: | The proposed methods explain variance in brain regions, especially in listening tasks, and the relative predictive power of the various syntax parsers is unexplored. |
Evaluating Pixel Language Models on Non-Standardized Languages (2025.coling-main)
Copied to clipboard
| Challenge: | pixel-based models can be used to transfer learning from standard languages to dialects . pretrained language models achieve strong results for languages seen during training, but their performance declines with out-of-domain dialects. |
| Approach: | They compare pixel-based models to token-based ones to evaluate dialects . standard german is tokenized in a more meaningful way, whereas the Bern dialect is tokenize in pixel form . |
| Outcome: | The proposed models outperform token-based models in part-of-speech tagging, dependency parsing and intent detection for zero-shot dialect evaluation by up to 26 percentage points in some scenarios, though not in Standard German. |
DisSent: Learning Sentence Representations from Explicit Discourse Relations (P19-1)
Copied to clipboard
| Challenge: | Existing models train on vast amounts of text or require costly, manually curated datasets. |
| Approach: | They propose to leverage the discourse relations between sentences to curate a high quality sentence relation task by leveraging explicit discourse relations. |
| Outcome: | The proposed model can be used to learn the meaning of two sentences in a bidirectional LSTM sentence encoder. |
Compositional Generalization in Dependency Parsing (2022.acl-long)
Copied to clipboard
| Challenge: | Compositionality is the ability to combine familiar units like words into novel phrases and sentences. |
| Approach: | They introduce a set of dependency parses for Compositional Freebase Queries (CFQ) they analyze the behaviour of a state-of-the-art dependency parser on the CFQ dataset . |
| Outcome: | The proposed dependency parser performs lower on the most challenging splits with the highest compound divergence. |
Representation Alignment and Adversarial Networks for Cross-lingual Dependency Parsing (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Pre-trained language models have improved dependency parsing accuracy in resource-rich languages . however, the accuracy drops sharply when the model is transferred to low-resource language . |
| Approach: | They propose a representation alignment and adversarial model to filter out useful knowledge from rich-resource language and ignore useless ones. |
| Outcome: | The proposed model outperforms baseline models on the benchmark datasets by 1.37 LAS and 1.34 UAS. |
Analyzing the Effect of Linguistic Similarity on Cross-Lingual Transfer: Tasks and Experimental Setups Matter (2025.findings-acl)
Copied to clipboard
| Challenge: | Prior work on cross-lingual transfer often focuses on a small set of languages from a few language families and/or a single task. |
| Approach: | They analyze cross-lingual transfer for 263 languages from a wide variety of language families . they include three popular NLP tasks: POS tagging, dependency parsing, topic classification . |
| Outcome: | The proposed approach is based on linguistic similarity measures for 263 languages . the results show that the effect of linguistic similarities on transfer performance depends on a range of factors . |
Improving Relation Extraction by Sequence-to-sequence-based Dependency Parsing Pre-training (2025.coling-main)
Copied to clipboard
| Challenge: | Existing studies show that dependency information is used only for encoder-only-based relation extraction tasks. |
| Approach: | They propose a syntax-aware seq2seq pre-trained model for relation extraction that incorporates dependency information into a seq2-trained language model by continual pre-training with a dependency parsing task. |
| Outcome: | The proposed model incorporates dependency information into a seq2seq pre-trained language model by continual pre-training with a generative sequence-to-sequence (sequ2sq)-based dependency parsing task. |
TwittIrish: A Universal Dependencies Treebank of Tweets in Modern Irish (2022.acl-long)
Copied to clipboard
| Challenge: | Modern Irish is a minority language lacking computational resources for accurate automatic syntactic parsing of user-generated content. |
| Approach: | They propose to use a treebank to facilitate natural language parsing of user-generated content in Irish. |
| Outcome: | The proposed treebank enables natural language processing of user-generated content in Irish. |
CSSL: Contrastive Self-Supervised Learning for Dependency Parsing on Relatively Free Word Ordered and Morphologically Rich Low Resource Languages (2024.emnlp-main)
Copied to clipboard
| Challenge: | Neural dependency parsing has achieved remarkable performance for low resource morphologically rich languages. |
| Approach: | They propose a self-supervised learning method to make the model robust to word order variations. |
| Outcome: | The proposed model shows a substantial gain of 3.03/2.95 points in 7 relatively free word order languages when compared to the best performing baseline. |
Enhancing Structure-aware Encoder with Extremely Limited Data for Graph-based Dependency Parsing (2022.coling-1)
Copied to clipboard
| Challenge: | Dependency parsing is an important natural language processing task which analyzes the syntactic structure of an input sentence. |
| Approach: | They propose a structure-aware encoder pre-trained on auto-parsed data to improve dependency parsing . they propose combining gold dependency trees with existing parsers to improve parser performance . |
| Outcome: | The proposed approach outperforms baselines under different parsers and dependency standards under different parameters and model architectures. |
Dual Graph Convolutional Networks for Aspect-based Sentiment Analysis (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods to model relationships between aspects and opinion words are inefficient due to informal expressions and complexity of online reviews. |
| Approach: | They propose a dual graph convolutional networks model that considers complementarity of syntax structures and semantic correlations simultaneously. |
| Outcome: | The proposed model outperforms state-of-the-art methods on three public datasets and validates it. |
Neural Aspect and Opinion Term Extraction with Mined Rules as Weak Supervision (P19-1)
Copied to clipboard
| Challenge: | Lack of labeled training data is a major bottleneck for aspect and opinion term extraction . et al., 2004: aspect and opinions are of particular importance for opinion mining . |
| Approach: | They propose to automatically mine extraction rules from existing training examples based on dependency parsing results . they then apply the mined rules to label auxiliary data to train a neural model . |
| Outcome: | The proposed algorithm can learn from human annotated data and human annnotated auxiliary data. |
Sequence Labeling Parsing by Learning across Representations (P19-1)
Copied to clipboard
| Challenge: | Constituency and dependency parsing are the main abstractions for representing syntactic structure of sentences . constituency parsers are considered disjointed tasks, and their improvements have been obtained separately. |
| Approach: | They propose to add auxiliary loss to constituency parsing paradigms and explore a model that parses both paradigms at no cost. |
| Outcome: | The proposed model outperforms single-task models by 1.05 F1 points and 0.62 UAS points for constituency parsing and dependency parsers. |
Probing for Labeled Dependency Trees (2022.acl-long)
Copied to clipboard
| Challenge: | Existing linear probes are limited to extracting undirected or unlabeled dependency parse trees, thereby capturing only a subset of the full task. |
| Approach: | They propose a linear probe which can extract labeled and directed dependency parse trees from embeddings while using fewer parameters and compute than prior methods. |
| Outcome: | The proposed method outperforms baselines and prior work in 13 languages and outperformed competitive baselines. |
Dependency Parsing via Sequence Generation (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for dependency parsing are transition-based, graph-based and sequence-to-sequence method. |
| Approach: | They propose to achieve dependency parsing (DP) via Sequence Generation (SG) by utilizing only the pre-trained language model without any auxiliary structures. |
| Outcome: | The proposed method performs well on DP benchmarks including PTB, UD2.2, SDP15 and SemEval16. |
Enough Is Enough! a Case Study on the Effect of Data Size for Evaluation Using Universal Dependencies (2024.lrec-main)
Copied to clipboard
| Challenge: | a recent study suggests that large datasets require large amounts of simulated data and scores to perform model evaluation. |
| Approach: | They propose to use dependency parsing to optimize annotation efforts by using a sample size of 5,000 tokens for in-language in-domain datasets. |
| Outcome: | The proposed method can be applied to other tasks, including classification and translation tasks. |
Linguistically-Informed Self-Attention for Semantic Role Labeling (D18-1)
Copied to clipboard
| Challenge: | Existing models of semantic role labeling use no explicit linguistic features. prior work has shown that syntax trees can dramatically improve SRL decoding. |
| Approach: | They propose a neural network model that incorporates syntax using only raw tokens . they show that LISA out-performs the state-of-the-art with contextually-encoded word representations a 1.0 F1 on newswire and 2.0 F1 in out-of domain text . |
| Outcome: | The proposed model outperforms the state-of-the-art model with word embeddings and predicted predicates. |
Span-based Semantic Role Labeling as Lexicalized Constituency Tree Parsing (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing models for semantic role labeling fail to capture the relationship between syntax and semantics. |
| Approach: | They propose a lexicalized tree representation for span-based SRL that integrates constituency and dependency parsing to explicitly model predicate-argument structures. |
| Outcome: | The proposed model achieves competitive performance on standard English benchmarks. |
Evaluating and Enhancing the Robustness of Neural Network-based Dependency Parsing Models with Adversarial Examples (2020.acl-main)
Copied to clipboard
| Challenge: | Previously studies focused on semantic tasks such as sentiment analysis, question answering and reading comprehension. |
| Approach: | They propose two approaches to study where and how adversarial examples exist in dependency parsing . they use a state-of-the-art parser to find adversarials in existing texts . |
| Outcome: | The proposed approaches show that adversarial examples exist in dependency parsing . they show that up to 77% of input examples admit adversarials . |
Transition-based Semantic Dependency Parsing with Pointer Networks (2020.acl-main)
Copied to clipboard
| Challenge: | Existing dependency parsers cannot be directly applied, so they need to be adaptable to deal with the absence of singlehead and connectedness constraints. |
| Approach: | They propose a transition system that produces labelled directed acyclic graphs and performs semantic dependency parsing with Pointer Networks. |
| Outcome: | The proposed system outperforms graph-based models and outperformed existing models on a harder NLP problem. |
Parsing as Tagging (2020.lrec-1)
Copied to clipboard
| Challenge: | Existing methods for dependency parsing treat parse as tagging, but they are not perfect. |
| Approach: | They propose a simple yet accurate method that treats parsing as tagging . they use a sequence model with a bidirectional LSTM over BERT embeddings . |
| Outcome: | The proposed method outperforms the state-of-the-art method on universal dependency (UD) by 1.76% unlabeled attachment score (UAS) for English, 1.98% UAS for French, and 1.16% UAS in German. |
CamemBERT: a Tasty French Language Model (2020.acl-main)
Copied to clipboard
Louis Martin, Benjamin Muller, Pedro Javier Ortiz Suárez, Yoann Dupont, Laurent Romary, Éric de la Clergerie, Djamé Seddah, Benoît Sagot
| Challenge: | Pretrained language models are now ubiquitous in Natural Language Processing, but their use in other languages is limited. |
| Approach: | They propose to train monolingual Transformer-based model for other languages using web crawled data instead of Wikipedia data and a relatively small web crawl dataset leads to better results. |
| Outcome: | The proposed model performs as well as those obtained using larger datasets. |
BERTie Bott’s Every Flavor Labels: A Tasty Introduction to Semantic Role Labeling for Galician (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing corpora, WordNet, and dependency parsing are used to build a semantic role labeling system. |
| Approach: | They use existing corpora, WordNet, and dependency parsing to build a Galician dataset for training semantic role labeling systems. |
| Outcome: | The proposed model outperforms the 2009 CoNLL Shared Task by 0.83 on Spanish datasets. |
Enhancing Discourse Dependency Parsing with Sentence Dependency Parsing: A Unified Generative Method Based on Code Representation (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Existing annotation resources for Discourse Dependency Parsing tasks are limited due to their complexity and annotation schema differences. |
| Approach: | They propose a code-based unified dependency parsing method that uses code to model dependency parses under different annotation schemas. |
| Outcome: | The proposed method improves on two Chinese DDP tasks. |
Revisiting Tri-training of Dependency Parsers (2021.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained word embeddings and self-training have been used in dependency parsing tasks for years. |
| Approach: | They compare tri-training and pretrained word embeddings in dependency parsing . they use language-specific FastText and ELMo embedds and multilingual BERT embedders . |
| Outcome: | The proposed methods are tri-training and pretrained word embeddings. |
Automatic Correction of Syntactic Dependency Annotation Differences (2022.lrec-1)
Copied to clipboard
| Challenge: | Annotation inconsistencies between data sets can cause problems for low-resource NLP . a simple method for automatically detecting annotation mismatches between corpora is proposed . |
| Approach: | They propose a method for automatically detecting annotation mismatches between dependency parsing corpora . they also propose three related methods for automatically configuring the mismatch . |
| Outcome: | The proposed method improves performance on both converted and unconverted data. |
Annotating the Tweebank Corpus on Named Entity Recognition and Building NLP Models for Social Media Analysis (2022.lrec-1)
Copied to clipboard
| Challenge: | Social media data such as Twitter messages pose a particular challenge to NLP systems because of their short, noisy nature. |
| Approach: | They create a Twitter-based NER corpus and train Tweet NLP models on it . they annotate named entities in TB2 using Amazon Mechanical Turk . |
| Outcome: | The proposed model outperforms existing models on Twitter and other social media platforms. |
CQE: A Comprehensive Quantity Extractor (2023.emnlp-main)
Copied to clipboard
| Challenge: | Quantities are essential in documents to describe factual information. |
| Approach: | They propose a comprehensive quantity extraction framework that detects combinations of values and units, the behavior of a quantity and the concept a quantity is associated with. |
| Outcome: | The proposed framework outperforms existing methods and is the first to detect concepts associated with identified quantities. |
NLPre: A Revised Approach towards Language-centric Benchmarking of Natural Language Preprocessing Systems (2024.lrec-main)
Copied to clipboard
| Challenge: | GLUE benchmarking system enables ongoing evaluation of multiple NLPre tools while credibly tracking their performance. |
| Approach: | They propose a language-centric benchmarking system that enables ongoing evaluation of multiple NLPre tools while credibly tracking their performance. |
| Outcome: | The proposed system is configured for Polish and integrated with the thoroughly assembled NLPre-PL benchmark. |
GRaMPa: Subword Regularisation by Skewing Uniform Segmentation Distributions with an Efficient Path-counting Markov Model (2025.acl-long)
Copied to clipboard
| Challenge: | Subword regularisations are known to be stochastic, but only a handful of possible segmentations are sampled. |
| Approach: | They propose to randomise word segmentations from a subword tokeniser instead of randomising them by weighting paths in an unweighted segmentation graph. |
| Outcome: | The proposed method outperforms existing methods on token-level tasks with spelling errors. |
Relation between Cross-Genre and Cross-Topic Transfer in Dependency Parsing (2024.lrec-main)
Copied to clipboard
| Challenge: | LAS scores in cross-genre transfer within and across treebanks align with topic distances. |
| Approach: | They propose to use topic modelling to assess whether genre is stable in dependency parsing with respect to topic distribution. |
| Outcome: | The proposed method is highly likely to capture topic similarity, but it is not clear whether it captures only genre features. |
Dependency Parsing-Based Syntactic Enhancement of Relation Extraction in Scientific Texts (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a pipeline approach to extract entities and relations from scientific text is challenging due to long sentences with densely packed entities. |
| Approach: | They propose a syntactic filtering method that prunes unlikely entity pairs before relation prediction. |
| Outcome: | The proposed method improves Rel+ F1 scores on SciERC, SciER, and ACE05 datasets. |
A Syntactic and Semantic Probe into Language Evolution based on Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on language evolution have relied on manual annotated resources and rely on dependency parsing. |
| Approach: | They propose to use attention-based structural distance and semantic space distance to measure language development. |
| Outcome: | The proposed measures show that human and LLMs share common characteristics in language processing. |